[% setvar title Transaction-enabled variables for Perl6 %]
<div id="archive-notice">
    <h3>This file is part of the Perl 6 Archive</h3>
    <p>To see what is currently happening visit <a href="http://www.perl6.org/">http://www.perl6.org/</a></p>
</div>
<div class='pod'>
<a name='TITLE'></a><h1>TITLE</h1>
<p>Transaction-enabled variables for Perl6</p>
<a name='VERSION'></a><h1>VERSION</h1>
<pre>  Maintainer: Szabó, Balázs &lt;<a href='mailto:dlux@kapu.hu'>dlux@kapu.hu</a>&gt;
  Date: 17 Aug 2000
  Last Modified: 13 Sep 2000
  Mailing List: <a href='mailto:perl6-internals@perl.org'>perl6-internals@perl.org</a>
  Number: 130
  Version: 6
  Status: Developing</pre>
<a name='ABSTRACT'></a><h1>ABSTRACT</h1>
<p>Transactions are quite important in a database-enabled application.
Professional database systems have transaction-handling inside, but there are
only a few laguage out there, what supports transactions in variable level.</p>
<p>In perl6, we can support transaction-enabled variables (including objects and
tied variables), and we can control transaction-enabled perl modules with that
(this include modules that do external I/O also). We can use our perl program
to tie several transaction-enabled data sources, and we can use perl to easily
maintain the consistency between them.</p>
<p>In this RFC we will look at how these variables would look like in perl6.</p>
<a name='STATUS'></a><h1>STATUS</h1>
<p>The idea and the implementation issuse is mainly clarified: We have now a tidy
and simple design.</p>
<p>Some questions are still open, these are the following:</p>
<ul>
<li><a name='Now we need to get a name for a new keyword name, which is more widely accepted than the current &quot;trans&quot;.'></a>Now we need to get a name for a new keyword name, which is more widely accepted
than the current &quot;trans&quot;.</li>
<li><a name='We need to decide whether we want multi-version concurrency control or not in the core. If we doesn't implement it, all things are remains simple. If we implement this, we will get smaller chance to deadlock, but it makes the core complicated.'></a>We need to decide whether we want multi-version concurrency control or not in
the core. If we doesn't implement it, all things are remains simple. If we
implement this, we will get smaller chance to deadlock, but it makes the core
complicated.</li>
<li><a name='We need to think about what if 'trans' is used with a non-transaction-enabled tied array or hash. How we can make it effective withour sacrificing either simplicity, speed or memory-requirement.'></a>We need to think about what if 'trans' is used with a non-transaction-enabled
tied array or hash. How we can make it effective withour sacrificing either
simplicity, speed or memory-requirement.</li>
</ul>
<a name='WHAT'S NEW IN VERSION 6'></a><h1>WHAT'S NEW IN VERSION 6</h1>
<pre>=over4 </pre>
<li><a name='Added STATUS section'></a>Added STATUS section</li>
<li><a name='Added some Implementation issues'></a>Added some Implementation issues</li>
<li><a name='Added some more explanation to the ABSTRACT section.'></a>Added some more explanation to the ABSTRACT section.</li>
</ul>
<a name='DESCRIPTION'></a><h1>DESCRIPTION</h1>
<p>In short, we have &quot;local&quot; keyword, which changes a value of a variable for only
the runtime of the current scope. The transaction-enabled variables should
start up like &quot;local&quot;, but IF the currenct scope reaches at the end, it then
copied into the global one.</p>
<p>We need to get a keyword to mark a variable transaction-enabled. I chosed
&quot;trans&quot; in this ducument, but other suggestions are welcome. Possible
alternatives are:</p>
<pre>  transaction
  transactional
  acid
  atomic
  onsuccess
  consistent</pre>
<p>The final decision  will be made by the porters, I use &quot;trans&quot; in this
document.</p>
<p>Preferred syntax:</p>
<pre>  sub trans_test { my ($self,@params)=@_;
    trans $self-&gt;{value}=$new_value;
  
    # ...
  
    die &quot;Error occured&quot; if $some_error;
  
    function_call(...)
  
    # ...
  
  } ;</pre>
<p>Meaning (in semi perl5 syntax):</p>
<pre>  sub trans_test {
    local $self-&gt;{value}=$new_value;
  
    # ...
  
    die &quot;Error occured&quot; if $ome_erre;
  
    function_call(...)
  
    # ...
  
    global $self-&gt;{value}=$self-&gt;{value};
  };</pre>
<p>If we want to gain more control and want to maintain easy syntax, we can use
another pragma, which sets up the attributes of the isolation of transaction
data.  I think the &quot;transaction&quot; pragma could be a good name:</p>
<pre>  use transaction (mode =&gt; 'lock', timeout=&gt;6000);</pre>
<p>Parameters for &quot;use transaction&quot;:</p>
<ul>
<li><a name='mode'></a>mode</li>
<p>can be:</p>
<ul>
<li><a name='simple'></a>simple</li>
<p>No blocking, concurrency control (default).</p>
<p>In a not-threaded environment this causes minimal overhead, and no locking
overhead at all.</p>
<li><a name='lock'></a>lock</li>
<p>Explicitly lock the accessed variables. (shared and exclusive locks used
between threads).</p>
<li><a name='version'></a>version</li>
<p>This is simlar to the postgres' multi-version concurrency control. It requires
more memory, but has a less chance to get into deadlock.</p>
</ul>
<li><a name='timeout'></a>timeout</li>
<p>Timeout in ms until we wait for the lock. 0 means nonblocking. If the timeout
reached, the system throws an exception.</p>
<li><a name='isolation'></a>isolation</li>
<p>Transaction isolation level. This can be:</p>
<ul>
<li><a name='0'></a>0</li>
<p>Read uncommitted</p>
<li><a name='1'></a>1</li>
<p>Read committed (default)</p>
<li><a name='2'></a>2</li>
<p>Repeatable read</p>
<li><a name='3'></a>3</li>
<p>Serializable.</p>
</ul>
<p>PostgreSQL implements only 1 and 3 AFAIK, so I think we could implment only
those levels. Then 0 and 2 will be equal to 1 and 3, but we could keep the
place for a future implementation.</p>
<p>See the postgreSQL documentation for the details.</p>
</ul>
<a name='Two phase commit'></a><h2>Two phase commit</h2>
<p>Two phase commit is the common way to deal with distributed transactions. Perl
need an interface to objects and tied variables to deal with these to become a
reliable transaction-handler. You can choose to implement these features in
your object and your tied variable. If you don't do that, perl will give you a
rough default.</p>
<p>At the end of the transaction, 2 different thing can happen: rollback or
commit. When rollback occured, all the transaction variables must be rolled
back. In commit, a two-phase commit procedure has been started.</p>
<p>The first phase is preparing to the commit: check the resources, allocates
resources to the commit, flushes caches, etc. After that it can decide wheter
you can do a commit or not. If all participants send &quot;yes&quot;, then the commit
phase begins: the coordinator sends &quot;commit&quot; messages to the participants, and
the transaction finishes. If any of the participants in the &quot;prepare&quot; phase
sends a false value, then the whole transaction need to be rolled back.</p>
<p>How it looks like in perl?</p>
<p>You have objects. Objects can be transaction-enabled, and if you want that, you
need to define the following functions as callbacks: COMMIT, ROLLBACK, PREPARE,
BEGIN_TRANSACTION. If you have a tied variable, then you can define callbacks
for this: TIE_COMMIT, TIE_ROLLBACK, TIE_PREPARE, TIE_BEGIN_TRANSACTION. These
can be used to extend an object or a tied variable to transaction-safe. If you
don't define PREPARE or TIE_PREPARE, then it will be only a one phase commit.
If you don't define COMMIT (or TIE_COMMIT) and ROLLBACK (or TIE_ROLLBACK), then
perl will do the simple &quot;copy back the old value on rollback&quot; mechanism, which
works well in cases when no multithreading and no special handling is necessary
for the data. If you don't define BEGIN_TRANSACTION or TIE_BEGIN_TRANSACTION,
then no special initialization performed on &quot;trans&quot; call.</p>
<a name='Tie interface'></a><h2>Tie interface</h2>
<p>Adding transaction-enabled property of a tied variable is not straightforward.
Imagine you have been tied a hash into a (not transaction-enabled) dbm file.
When you fetch, you need to put a shared lock (or version-control) the dbm file
or key, when you read, you need to put an exclusive lock, and when the
transaction ends, you need to release the lock. For this reason, we can add two
callback: TIE_COMMIT and TIE_ROLLBACK.</p>
<p>If we don't want to use locking, or want to do an advanced
transaction-management, we can provide a transaction-id to the callbacks. This
can be done with a new package global variable (which is localized in every
call), the name can be  $Package::TRANSACTION_ID. We could use a new parameter,
but it is not is not so neat, because some of the callbacks (PUSH, POP,
UNSHIFT, PRINT, PRINTF, etc) are expecting LISTs as an attribute, and this can
cause unnecessary rewrite of the tie interface.</p>
<p>Following is the description of the modifications of the tie interface:</p>
<ul>
<li><a name='New package global'></a>New package global</li>
<p>$Package::TRANSACTION_ID will be a unique identifier of the current transaction
(if any).</p>
<li><a name='New Callbacks'></a>New Callbacks</li>
<p>Some new callbacks required:</p>
<ul>
<li><a name='TIE_PREPARE $this'></a>TIE_PREPARE $this</li>
<p>This is the first part of the 2 way commit transaction. This must return true
if the variable is prepared for the COMMIT, false otherwise. If this callback
is not defined, then the variable lose the right to abort the transaction, and
perl implicitly returns 1 in this cases.</p>
<li><a name='TIE_COMMIT $this'></a>TIE_COMMIT $this</li>
<p>If it is defined, then it is called after TIE_PREPARE returns 1 for all the
transaction-enabled variables in the current scope. This must be used to commit
the transaction.</p>
<li><a name='TIE_ROLLBACK $this'></a>TIE_ROLLBACK $this</li>
<p>If it is defined, then it is called at the end of a failed transaction. If NOT
defined, then STORE will be called with the old value of the variable.</p>
<li><a name='TIE_BEGIN_TRANSACTION $this,$parent_transaction_id'></a>TIE_BEGIN_TRANSACTION $this,$parent_transaction_id</li>
<p>It is called by the system, if the program execution finds a &quot;trans $this&quot; in
the program. This is intended to initialize the transaction.</p>
<p>This callback is special, because if you use the &quot;trans&quot; or &quot;local&quot; keyword in
this subroutine, this will not last only the end of the sub, but the end of the
transaction!</p>
<p>$parent_transaction_id can be used to track the sub-transaction relations. This
is the ID of the transaction which contains this transaction. undef if this is
the master transaction (has no parent). If a subtransaction is terminated (and
the exception is catched), then the proper rollback callbacks are called, but
the parent transaction can be successful!</p>
</ul>
</ul>
<p>If a package used in &quot;tie&quot; has one of the above callbacks, then perl _must_
emulate the transaction in every call, so a simple FETCH in non-transaction
enironment must be the sequence of TIE_BEGIN_TRANSACTION, FETCH, TIE_PREPARE ?
TIE_COMMIT : TIE_ROLLBACK and a simple STORE must be: TIE_BEGIN_TRANSACTION,
STORE, TIE_PREPARE ? TIE_COMMIT : TIE_ROLLBACK.</p>
<a name='Object interface'></a><h2>Object interface</h2>
<p>Object interface is similar to the tied interface: you will need callbacks:
PREPARE, COMMIT, ROLLBACK and BEGIN_TRANSACTION. These will do the same as
described in the Tie interface. The $Package::TRANSACTION_ID will be set in
this case also.</p>
<p>Note, if you declare an object as &quot;trans&quot;, this means that this is localized
for the runtime of the transaction and that PREPARE, COMMIT, ROLLBACK will be
called at the end of the block of the declaration. It doesn't mean that all the
data structure under that is transaction safe. It cannot be guaranteed, and you
need to explicitly declare them as &quot;trans&quot; variables.</p>
<a name='TRANSACTION-ENABLED TIED VARIABLE EXAMPLE'></a><h1>TRANSACTION-ENABLED TIED VARIABLE EXAMPLE</h1>
<p>This is an example of a transaction-enabled tie interface.</p>
<p>The following package can be tied to any variable, and can be used as a
persistent, transaction-enabled data.</p>
<p>Usage:</p>
<pre>  tie $scalar, &quot;Transaction::ScalarFile&quot;, $filename;

  sub my_transaction {
    trans $scalar;
    $scalar=&quot;Perl&quot; x 1024;

    ...

  };</pre>
<p>The data in the file referred by $filename can be accessed, modified as
$scalar. $scalar can be used in a transaction, supports subtransactions, and
supports two-phase commits and locks the accessed file with flock(), so it can
be used in multithreaded and multiprocess environment.</p>
<p>Here is the code:</p>
<pre>  package Transaction::ScalarFile;
  use transaction (mode =&gt; 'lock', timeout =&gt; 30);
  use strict;
  use Fcntl qw( :flock );

  # constant declaration
  sub FILENAME        { 0; };
  sub FILEHANDLE      { 1; };
  sub VALUE           { 2; };
  sub LOCKED          { 3; };
  sub PARENT_TRANS    { 4; };
  sub TEMP_FILENAME   { 5; };
  sub TEMP_FILEHANDLE { 6; };

  sub TIE_BEGIN_TRANSACTION { my ($s,$parent)=@_;
    trans $s-&gt;[VALUE];  # The value is transaction-enabled
    local $s-&gt;[PARENT_TRANS]=$parent;
  };

  sub TIESCALAR { my ($class,$filename)=@_;
    my $s=[$filename];
    bless $s,ref($class) || $class;
    $s;
  };

  sub FETCH { my ($s)=@_;
    return $s-&gt;[VALUE] if defined $s-&gt;[VALUE];
    $s-&gt;open or return undef;
    flock $s-&gt;[FILEHANDLE], LOCK_SH;
    local $/=undef;
    $s-&gt;[LOCKED]=1;
    return $s-&gt;[VALUE]= &lt; $s-&gt;[FILEHANDLE] &gt;;
  };

  sub STORE { my ($s,$value)=@_;
    if ($s-&gt;[LOCKED]&lt;=1) {
      $s-&gt;open or return undef;
      flock $s-&gt;[FILEHANDLE], LOCK_EX;
      $s-&gt;[LOCKED]=2;
    };
    $s-&gt;[VALUE]=$value;
  };

  sub TIE_PREPARE { my ($s)=@_;
    # If it is a subtransaction: do nothing
    return 1 if $s-&gt;[PARENT_TRANS]; 
    # If the value is not modified: do nothing
    return 1 if $s-&gt;[LOCKED]&lt;2;     
    # Transaction failed if I cannot make the temp file
    $s-&gt;create_and_lock_temp_file or return 0;
    $!=0; # reset ERRNO
    # Writes the new value to the tempfile
    print $s-&gt;[TEMP_FILEHANDLE], $s-&gt;[VALUE];
    # Transaction failed if error occured
    unlink($s-&gt;[TEMP_FILENAME]),return 0 if $!;
    return 1;
  };

  sub TIE_COMMIT { my ($s)=@_;
    return if $s-&gt;[PARENT_TRANS];
    # This is the weakest point of the transaction, we cannot make those two
    # operations atomic ...
    rename $s-&gt;[FILENAME], $s-&gt;[FILENAME].&quot;.old.$$&quot;;
    rename $s-&gt;[TEMP_FILENAME],$s-&gt;[FILENAME];
    unlink $s-&gt;[FILENAME].&quot;.old.$$&quot;;
    flock $s-&gt;[FILEHANDLE],LOCK_UN;
    flock $s-&gt;[TEMP_FILEHANDLE],LOCK_UN;
    $s-&gt;[LOCKED]=0;
    $s-&gt;[TEMP_FILEHANDLE] = $s-&gt;[TEMP_FILENAME] = undef;
  };

  sub TIE_ROLLBACK { my ($s)=@_;
    return if $s-&gt;[PARENT_TRANS]; # We don't care if it has parent trans.
    flock $s-&gt;[TEMP_FIELHANDLE], LOCK_UN;
    flock $s-&gt;[FILEHANDLE], LOCK_UN;
    unlink $s-&gt;[TEMP_FILENAME];
    $s-&gt;[TEMP_FILEHANDLE] = $s-&gt;[TEMP_FILENAME] = undef;
  };

  sub open { my ($s)=@_;
    return $s-&gt;[FILEHANDLE]=new FileHandle(&quot;&lt;&quot;.$s-&gt;[FILENAME]);
  };

  sub create_and_lock_temp_file { my ($s)=@_;
    $s-&gt;[TEMP_FILENAME]=$s-&gt;[FILENAME].&quot;.trans.$$&quot;;
    $s-&gt;[TEMP_FILEHANDLE]=new FileHandle(&quot;&gt;&quot;.$s-&gt;[TEMP_FILENAME) or return 0;
    flock $s-&gt;[TEMP_FILEHANDLE],LOCK_EX;
  };</pre>
<a name='IMPLEMENTATION'></a><h1>IMPLEMENTATION</h1>
<a name='Transaction handling methods'></a><h2>Transaction handling methods</h2>
<ul>
<li><a name='simple'></a>simple</li>
<p>This is the default method. This needs no magic, implementation is
straightforward:</p>
<p>When you use &quot;trans&quot;, then the transaction-enabled variable is duplicated
automatically in the memory (with a copy-on-write method). IF the transaction
succeeded, this will copied back to the original.</p>
<li><a name='lock'></a>lock</li>
<p>We need to maintain locks (mutexes) on variables. We assume this will be used
in threaded applications.</p>
<p>When we use &quot;trans&quot;, then perl will put a shared lock on the variable.</p>
<p>When we read the variable, we also put a shared lock to that.</p>
<p>When we write the variable, we check if it is already locked, and if we locked
that already or no exclusive locks present, then write to the value, and lock
that with LOCK_EX. If other exclusive lock present on the variable, then we
need to wait for the releasing.</p>
<p>When the &quot;trans&quot; content ends, we frees the shared (or exclusive lock).  If the
content ends with a die then we puts the original value back if we have locked
it with exclusive lock.</p>
<li><a name='version'></a>version</li>
<p>It is the mechanism of making multiple versioned copies of the variable every
time somebody access this. This needs tiestamping, and postgreSQL-like
concurrency control. I don't know more details.</p>
</ul>
<a name='Use 'trans' for non-transaction-enabled tied vars and object'></a><h2>Use 'trans' for non-transaction-enabled tied vars and object</h2>
<p>If we use 'trans' keyword for a value which is a tied variable or an object,
but which doesn't implement the transaction-interface, our transaction-safe
environment is not guaranteed to be consistent anymore. We cannot make the
system 100% transaction-safe anymore. All we can do is to emulate the
transactional behaviour with our current tools.</p>
<p>If we take a look at the TIE interface, then we can emulate the transaction
behaviour only with STORE and FETCH. It is not a problem with a simple scalar,
but it is a problem if we think about a tied array or a tied hash, or simply we
throw a fatal exception if someone try this. We must think about it.</p>
<p>One point is clear: the transaction is as weak as the weakest
transaction-enabled variable in it no matter how we emulate the
transaction-behaviour.</p>
<a name='CHANGES IN PREVIOUS VERSIONS'></a><h1>CHANGES IN PREVIOUS VERSIONS</h1>
<a name='Version 5'></a><h2>Version 5</h2>
<ul>
<li><a name='TRANSACTION-ENABLED TIED VARIABLE EXAMPLE added'></a>TRANSACTION-ENABLED TIED VARIABLE EXAMPLE added</li>
<li><a name='BEGIN_TRANSACTION and TIE_BEGIN_TRANSACTION added'></a>BEGIN_TRANSACTION and TIE_BEGIN_TRANSACTION added</li>
<li><a name='TIE* renamed to TIE_*'></a>TIE* renamed to TIE_*</li>
<li><a name='CHANGES moved to the end of the RFC'></a>CHANGES moved to the end of the RFC</li>
<li><a name='Added alternatives to &quot;trans&quot;'></a>Added alternatives to &quot;trans&quot;</li>
</ul>
<a name='Version 4'></a><h2>Version 4</h2>
<ul>
<li><a name='TIE interface callbacks are renamed to TIE_COMMIT, TIE_ROLLBACK'></a>TIE interface callbacks are renamed to TIE_COMMIT, TIE_ROLLBACK</li>
<li><a name='2 phase commit described'></a>2 phase commit described</li>
<li><a name='PREPARE, TIE_PREPARE added to support two phase commits'></a>PREPARE, TIE_PREPARE added to support two phase commits</li>
<li><a name='Object interface described'></a>Object interface described</li>
<li><a name='&quot;safe&quot; renamed to &quot;trans&quot;'></a>&quot;safe&quot; renamed to &quot;trans&quot;</li>
</ul>
<a name='Version 3'></a><h2>Version 3</h2>
<ul>
<li><a name='Added a tie interface change request: COMMIT and ROLLBACK, and new global'></a>Added a tie interface change request: COMMIT and ROLLBACK, and new global</li>
<li><a name='Fixed some Formatting error of this pod.'></a>Fixed some Formatting error of this pod.</li>
<li><a name=''use varlock' renamed to 'use transaction'.'></a>'use varlock' renamed to 'use transaction'.</li>
</ul>
<a name='Version 2'></a><h2>Version 2</h2>
<ul>
<li><a name='Detailed implementation description'></a>Detailed implementation description</li>
<li><a name='Add a new pragma 'varlock' for controlling the concurrency control.'></a>Add a new pragma 'varlock' for controlling the concurrency control.</li>
</ul>
<a name='REFERENCES'></a><h1>REFERENCES</h1>
<p>PostgreSQL Multi-version concurrency control
<a href='http://www.postgresql.org/docs/postgres/mvcc.htm' target='_blank'>www.postgresql.org</a></p>
<p>Two phase commit: (Google found that :-)
<a href='http://oradoc.photo.net/ora8doc/DOC/server803/A54653_01/ds_ch3.htm' target='_blank'>oradoc.photo.net</a></p>
<p>RFC 19: Rename the <code>local</code> operator</p>
<p>RFC 119: object neutral error handling via exceptions</p>
<p>perldoc perlthread: the perl5 threading interface</p>
<p>perldoc perltie: the perl5 tie interface</p>
</div>
